{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Predictive Maintenance\n",
    "## Fault Classification using Supervised learning\n",
    "\n",
    "#### Author Nagdev Amruthnath\n",
    "Date: 1/10/2019\n",
    "\n",
    "##### Citation Info\n",
    "If you are using this for your research, please use the following for citation. \n",
    "\n",
    "Amruthnath, Nagdev, and Tarun Gupta. \"A research study on unsupervised machine learning algorithms for early fault detection in predictive maintenance.\" In 2018 5th International Conference on Industrial Engineering and Applications (ICIEA), pp. 355-361. IEEE, 2018.\n",
    "\n",
    "##### Disclaimer\n",
    "This is a tutorial for performing fault detection using machine learning. You this code at your own risk. I do not gurantee that this would work as shown below. If you have any suggestions please branch this project.\n",
    "\n",
    "## Introduction\n",
    "This is the first of four part demostration series of using machine learning for predictive maintenance.   \n",
    "\n",
    "The area of predictive maintenance has taken a lot of prominence in the last couple of years due to various reasons. With new algorithms and methodologies growing across different learning methods, it has remained a challenge for industries to adopt which method is fit, robust and provide most accurate detection. One the most common learning approaches used today for fault diagnosis is supervised learning. This is wholly based on the predictor variable and response variable. In this tutorial, we will be looking into some of the common supervised learning models such as SVM, random forest, k-nearest neighbour and H2O's AutoML model.\n",
    "\n",
    "\n",
    "## Load libraries "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T19:23:37.732876Z",
     "start_time": "2019-01-10T19:23:37.657Z"
    }
   },
   "outputs": [],
   "source": [
    "options(warn=-1)\n",
    "\n",
    "# load libraries\n",
    "library(mdatools) #mdatools version 0.9.1\n",
    "library(caret)\n",
    "library(foreach)\n",
    "library(dplyr)\n",
    "library(h2o)\n",
    "library(doParallel, verbose = F)\n",
    "library(ModelMetrics, verbose = F)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load data\n",
    "Here we are using data from a bench press. There are total of four different states in this machine and they are split into four different csv files. We need to load the data first. In the data time represents the time between samples, ax is the acceleration on x axis, ay is the acceleration on y axis, az is the acceleration on z axis and at is the G's. The data was collected at sample rate of 100hz.   \n",
    "\n",
    "Four different states of the machine were collected  \n",
    "1. Nothing attached to drill press\n",
    "2. Wooden base attached to drill press\n",
    "3. Imbalance created by adding weight to one end of wooden base\n",
    "4. Imbalacne created by adding weight to two ends of wooden base."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T19:23:40.522397Z",
     "start_time": "2019-01-10T19:23:40.029Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>time</th><th scope=col>ax</th><th scope=col>ay</th><th scope=col>az</th><th scope=col>aT</th><th scope=col>y</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>0.002  </td><td>-0.3246</td><td> 0.2748</td><td> 0.1502</td><td>0.451  </td><td>1      </td></tr>\n",
       "\t<tr><td>0.009  </td><td> 0.6020</td><td>-0.1900</td><td>-0.3227</td><td>0.709  </td><td>1      </td></tr>\n",
       "\t<tr><td>0.019  </td><td> 0.9787</td><td> 0.3258</td><td> 0.0124</td><td>1.032  </td><td>1      </td></tr>\n",
       "\t<tr><td>0.027  </td><td> 0.6141</td><td>-0.4179</td><td> 0.0471</td><td>0.744  </td><td>1      </td></tr>\n",
       "\t<tr><td>0.038  </td><td>-0.3218</td><td>-0.6389</td><td>-0.4259</td><td>0.833  </td><td>1      </td></tr>\n",
       "\t<tr><td>0.047  </td><td>-0.3607</td><td> 0.1332</td><td>-0.1291</td><td>0.406  </td><td>1      </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|llllll}\n",
       " time & ax & ay & az & aT & y\\\\\n",
       "\\hline\n",
       "\t 0.002   & -0.3246 &  0.2748 &  0.1502 & 0.451   & 1      \\\\\n",
       "\t 0.009   &  0.6020 & -0.1900 & -0.3227 & 0.709   & 1      \\\\\n",
       "\t 0.019   &  0.9787 &  0.3258 &  0.0124 & 1.032   & 1      \\\\\n",
       "\t 0.027   &  0.6141 & -0.4179 &  0.0471 & 0.744   & 1      \\\\\n",
       "\t 0.038   & -0.3218 & -0.6389 & -0.4259 & 0.833   & 1      \\\\\n",
       "\t 0.047   & -0.3607 &  0.1332 & -0.1291 & 0.406   & 1      \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| time | ax | ay | az | aT | y |\n",
       "|---|---|---|---|---|---|\n",
       "| 0.002   | -0.3246 |  0.2748 |  0.1502 | 0.451   | 1       |\n",
       "| 0.009   |  0.6020 | -0.1900 | -0.3227 | 0.709   | 1       |\n",
       "| 0.019   |  0.9787 |  0.3258 |  0.0124 | 1.032   | 1       |\n",
       "| 0.027   |  0.6141 | -0.4179 |  0.0471 | 0.744   | 1       |\n",
       "| 0.038   | -0.3218 | -0.6389 | -0.4259 | 0.833   | 1       |\n",
       "| 0.047   | -0.3607 |  0.1332 | -0.1291 | 0.406   | 1       |\n",
       "\n"
      ],
      "text/plain": [
       "  time  ax      ay      az      aT    y\n",
       "1 0.002 -0.3246  0.2748  0.1502 0.451 1\n",
       "2 0.009  0.6020 -0.1900 -0.3227 0.709 1\n",
       "3 0.019  0.9787  0.3258  0.0124 1.032 1\n",
       "4 0.027  0.6141 -0.4179  0.0471 0.744 1\n",
       "5 0.038 -0.3218 -0.6389 -0.4259 0.833 1\n",
       "6 0.047 -0.3607  0.1332 -0.1291 0.406 1"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#read csv files\n",
    "file1 = read.csv(\"dry run.csv\", sep=\",\", header =T)\n",
    "file2 = read.csv(\"base.csv\", sep=\",\", header =T)\n",
    "file3 = read.csv(\"imbalance 1.csv\", sep=\",\", header =T)\n",
    "file4 = read.csv(\"imbalance 2.csv\", sep=\",\", header =T)\n",
    "\n",
    "#Add labels to data\n",
    "file1$y = 1\n",
    "file2$y = 2\n",
    "file3$y = 3\n",
    "file4$y = 4\n",
    "\n",
    "#view top rows of data\n",
    "head(file1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can look at the summary of each file using summary function in R. Below, we can observe that 66 seconds long data is available. We also have min, max and mean for each of the variables. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T19:23:42.029410Z",
     "start_time": "2019-01-10T19:23:41.958Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "      time              ax                 ay                  az         \n",
       " Min.   : 0.002   Min.   :-2.11880   Min.   :-2.143600   Min.   :-4.1744  \n",
       " 1st Qu.:16.507   1st Qu.:-0.41478   1st Qu.:-0.625250   1st Qu.:-0.7359  \n",
       " Median :33.044   Median : 0.02960   Median :-0.022050   Median :-0.1468  \n",
       " Mean   :33.037   Mean   : 0.01233   Mean   : 0.008697   Mean   :-0.1021  \n",
       " 3rd Qu.:49.535   3rd Qu.: 0.46003   3rd Qu.: 0.641700   3rd Qu.: 0.4298  \n",
       " Max.   :66.033   Max.   : 2.09620   Max.   : 2.003000   Max.   : 4.9466  \n",
       "       aT              y    \n",
       " Min.   :0.032   Min.   :1  \n",
       " 1st Qu.:0.848   1st Qu.:1  \n",
       " Median :1.169   Median :1  \n",
       " Mean   :1.277   Mean   :1  \n",
       " 3rd Qu.:1.579   3rd Qu.:1  \n",
       " Max.   :5.013   Max.   :1  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# summary of each file\n",
    "summary(file1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Aggregration and feature extraction\n",
    "Here, the data is aggregated by 1 minute and features are extracted. Features are extracted to reduce the dimension of the data and only storing the representation of the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T19:23:46.121459Z",
     "start_time": "2019-01-10T19:23:45.638Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>group</th><th scope=col>ax_mean</th><th scope=col>ax_sd</th><th scope=col>ax_min</th><th scope=col>ax_max</th><th scope=col>ax_median</th><th scope=col>ay_mean</th><th scope=col>ay_sd</th><th scope=col>ay_min</th><th scope=col>ay_may</th><th scope=col>...</th><th scope=col>az_sd</th><th scope=col>az_min</th><th scope=col>az_maz</th><th scope=col>az_median</th><th scope=col>aT_mean</th><th scope=col>aT_sd</th><th scope=col>aT_min</th><th scope=col>aT_maT</th><th scope=col>aT_median</th><th scope=col>y</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>0            </td><td>-0.038164706 </td><td>0.6594686    </td><td>-1.2587      </td><td>1.3821       </td><td>-0.0955      </td><td>-0.0682627451</td><td>0.7506785    </td><td>-1.3892      </td><td>1.6418       </td><td>...          </td><td>0.9845115    </td><td>-2.6753      </td><td>2.7507       </td><td> 0.0254      </td><td>1.273216     </td><td>0.5830149    </td><td>0.400        </td><td>3.029        </td><td>1.0770       </td><td>1            </td></tr>\n",
       "\t<tr><td>1            </td><td>-0.005806122 </td><td>0.6325808    </td><td>-1.6194      </td><td>1.1943       </td><td>-0.0015      </td><td> 0.0037908163</td><td>0.7819044    </td><td>-1.5625      </td><td>1.5428       </td><td>...          </td><td>0.9252188    </td><td>-3.0774      </td><td>2.7158       </td><td>-0.2121      </td><td>1.263622     </td><td>0.5448447    </td><td>0.410        </td><td>3.197        </td><td>1.1375       </td><td>1            </td></tr>\n",
       "\t<tr><td>2            </td><td> 0.069845455 </td><td>0.6665500    </td><td>-1.4554      </td><td>1.4667       </td><td> 0.1070      </td><td> 0.0744333333</td><td>0.8022922    </td><td>-1.4800      </td><td>1.7951       </td><td>...          </td><td>0.9293866    </td><td>-1.8205      </td><td>2.4862       </td><td>-0.1512      </td><td>1.298364     </td><td>0.5131552    </td><td>0.255        </td><td>2.644        </td><td>1.2830       </td><td>1            </td></tr>\n",
       "\t<tr><td>3            </td><td> 0.011552525 </td><td>0.5511310    </td><td>-1.9254      </td><td>1.2034       </td><td> 0.0675      </td><td> 0.0008262626</td><td>0.7894209    </td><td>-2.0042      </td><td>1.5577       </td><td>...          </td><td>0.8893505    </td><td>-2.1562      </td><td>3.2355       </td><td>-0.1672      </td><td>1.203848     </td><td>0.5125826    </td><td>0.393        </td><td>3.322        </td><td>1.1180       </td><td>1            </td></tr>\n",
       "\t<tr><td>4            </td><td> 0.046688119 </td><td>0.6426574    </td><td>-1.7805      </td><td>1.4837       </td><td> 0.0836      </td><td>-0.0177594059</td><td>0.7510811    </td><td>-1.6629      </td><td>1.4369       </td><td>...          </td><td>0.9265720    </td><td>-1.8515      </td><td>3.5451       </td><td>-0.1741      </td><td>1.226267     </td><td>0.5824608    </td><td>0.313        </td><td>3.597        </td><td>1.1720       </td><td>1            </td></tr>\n",
       "\t<tr><td>5            </td><td> 0.006678788 </td><td>0.5780957    </td><td>-1.4719      </td><td>1.4355       </td><td> 0.0536      </td><td> 0.0013626263</td><td>0.7812245    </td><td>-1.6293      </td><td>1.6362       </td><td>...          </td><td>0.9091516    </td><td>-2.5561      </td><td>2.9196       </td><td>-0.2588      </td><td>1.209515     </td><td>0.5664847    </td><td>0.336        </td><td>3.035        </td><td>1.1590       </td><td>1            </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|llllllllllllllllllllll}\n",
       " group & ax\\_mean & ax\\_sd & ax\\_min & ax\\_max & ax\\_median & ay\\_mean & ay\\_sd & ay\\_min & ay\\_may & ... & az\\_sd & az\\_min & az\\_maz & az\\_median & aT\\_mean & aT\\_sd & aT\\_min & aT\\_maT & aT\\_median & y\\\\\n",
       "\\hline\n",
       "\t 0             & -0.038164706  & 0.6594686     & -1.2587       & 1.3821        & -0.0955       & -0.0682627451 & 0.7506785     & -1.3892       & 1.6418        & ...           & 0.9845115     & -2.6753       & 2.7507        &  0.0254       & 1.273216      & 0.5830149     & 0.400         & 3.029         & 1.0770        & 1            \\\\\n",
       "\t 1             & -0.005806122  & 0.6325808     & -1.6194       & 1.1943        & -0.0015       &  0.0037908163 & 0.7819044     & -1.5625       & 1.5428        & ...           & 0.9252188     & -3.0774       & 2.7158        & -0.2121       & 1.263622      & 0.5448447     & 0.410         & 3.197         & 1.1375        & 1            \\\\\n",
       "\t 2             &  0.069845455  & 0.6665500     & -1.4554       & 1.4667        &  0.1070       &  0.0744333333 & 0.8022922     & -1.4800       & 1.7951        & ...           & 0.9293866     & -1.8205       & 2.4862        & -0.1512       & 1.298364      & 0.5131552     & 0.255         & 2.644         & 1.2830        & 1            \\\\\n",
       "\t 3             &  0.011552525  & 0.5511310     & -1.9254       & 1.2034        &  0.0675       &  0.0008262626 & 0.7894209     & -2.0042       & 1.5577        & ...           & 0.8893505     & -2.1562       & 3.2355        & -0.1672       & 1.203848      & 0.5125826     & 0.393         & 3.322         & 1.1180        & 1            \\\\\n",
       "\t 4             &  0.046688119  & 0.6426574     & -1.7805       & 1.4837        &  0.0836       & -0.0177594059 & 0.7510811     & -1.6629       & 1.4369        & ...           & 0.9265720     & -1.8515       & 3.5451        & -0.1741       & 1.226267      & 0.5824608     & 0.313         & 3.597         & 1.1720        & 1            \\\\\n",
       "\t 5             &  0.006678788  & 0.5780957     & -1.4719       & 1.4355        &  0.0536       &  0.0013626263 & 0.7812245     & -1.6293       & 1.6362        & ...           & 0.9091516     & -2.5561       & 2.9196        & -0.2588       & 1.209515      & 0.5664847     & 0.336         & 3.035         & 1.1590        & 1            \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| group | ax_mean | ax_sd | ax_min | ax_max | ax_median | ay_mean | ay_sd | ay_min | ay_may | ... | az_sd | az_min | az_maz | az_median | aT_mean | aT_sd | aT_min | aT_maT | aT_median | y |\n",
       "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|\n",
       "| 0             | -0.038164706  | 0.6594686     | -1.2587       | 1.3821        | -0.0955       | -0.0682627451 | 0.7506785     | -1.3892       | 1.6418        | ...           | 0.9845115     | -2.6753       | 2.7507        |  0.0254       | 1.273216      | 0.5830149     | 0.400         | 3.029         | 1.0770        | 1             |\n",
       "| 1             | -0.005806122  | 0.6325808     | -1.6194       | 1.1943        | -0.0015       |  0.0037908163 | 0.7819044     | -1.5625       | 1.5428        | ...           | 0.9252188     | -3.0774       | 2.7158        | -0.2121       | 1.263622      | 0.5448447     | 0.410         | 3.197         | 1.1375        | 1             |\n",
       "| 2             |  0.069845455  | 0.6665500     | -1.4554       | 1.4667        |  0.1070       |  0.0744333333 | 0.8022922     | -1.4800       | 1.7951        | ...           | 0.9293866     | -1.8205       | 2.4862        | -0.1512       | 1.298364      | 0.5131552     | 0.255         | 2.644         | 1.2830        | 1             |\n",
       "| 3             |  0.011552525  | 0.5511310     | -1.9254       | 1.2034        |  0.0675       |  0.0008262626 | 0.7894209     | -2.0042       | 1.5577        | ...           | 0.8893505     | -2.1562       | 3.2355        | -0.1672       | 1.203848      | 0.5125826     | 0.393         | 3.322         | 1.1180        | 1             |\n",
       "| 4             |  0.046688119  | 0.6426574     | -1.7805       | 1.4837        |  0.0836       | -0.0177594059 | 0.7510811     | -1.6629       | 1.4369        | ...           | 0.9265720     | -1.8515       | 3.5451        | -0.1741       | 1.226267      | 0.5824608     | 0.313         | 3.597         | 1.1720        | 1             |\n",
       "| 5             |  0.006678788  | 0.5780957     | -1.4719       | 1.4355        |  0.0536       |  0.0013626263 | 0.7812245     | -1.6293       | 1.6362        | ...           | 0.9091516     | -2.5561       | 2.9196        | -0.2588       | 1.209515      | 0.5664847     | 0.336         | 3.035         | 1.1590        | 1             |\n",
       "\n"
      ],
      "text/plain": [
       "  group ax_mean      ax_sd     ax_min  ax_max ax_median ay_mean       ay_sd    \n",
       "1 0     -0.038164706 0.6594686 -1.2587 1.3821 -0.0955   -0.0682627451 0.7506785\n",
       "2 1     -0.005806122 0.6325808 -1.6194 1.1943 -0.0015    0.0037908163 0.7819044\n",
       "3 2      0.069845455 0.6665500 -1.4554 1.4667  0.1070    0.0744333333 0.8022922\n",
       "4 3      0.011552525 0.5511310 -1.9254 1.2034  0.0675    0.0008262626 0.7894209\n",
       "5 4      0.046688119 0.6426574 -1.7805 1.4837  0.0836   -0.0177594059 0.7510811\n",
       "6 5      0.006678788 0.5780957 -1.4719 1.4355  0.0536    0.0013626263 0.7812245\n",
       "  ay_min  ay_may ... az_sd     az_min  az_maz az_median aT_mean  aT_sd    \n",
       "1 -1.3892 1.6418 ... 0.9845115 -2.6753 2.7507  0.0254   1.273216 0.5830149\n",
       "2 -1.5625 1.5428 ... 0.9252188 -3.0774 2.7158 -0.2121   1.263622 0.5448447\n",
       "3 -1.4800 1.7951 ... 0.9293866 -1.8205 2.4862 -0.1512   1.298364 0.5131552\n",
       "4 -2.0042 1.5577 ... 0.8893505 -2.1562 3.2355 -0.1672   1.203848 0.5125826\n",
       "5 -1.6629 1.4369 ... 0.9265720 -1.8515 3.5451 -0.1741   1.226267 0.5824608\n",
       "6 -1.6293 1.6362 ... 0.9091516 -2.5561 2.9196 -0.2588   1.209515 0.5664847\n",
       "  aT_min aT_maT aT_median y\n",
       "1 0.400  3.029  1.0770    1\n",
       "2 0.410  3.197  1.1375    1\n",
       "3 0.255  2.644  1.2830    1\n",
       "4 0.393  3.322  1.1180    1\n",
       "5 0.313  3.597  1.1720    1\n",
       "6 0.336  3.035  1.1590    1"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "file1$group = as.factor(round(file1$time))\n",
    "file2$group = as.factor(round(file2$time))\n",
    "file3$group = as.factor(round(file3$time))\n",
    "file4$group = as.factor(round(file4$time))\n",
    "#(file1,20)\n",
    "\n",
    "#list of all files\n",
    "files = list(file1, file2, file3, file4)\n",
    "\n",
    "#loop through all files and combine\n",
    "features = NULL\n",
    "for (i in 1:4){\n",
    "res = files[[i]] %>%\n",
    "    group_by(group) %>%\n",
    "    summarize(ax_mean = mean(ax),\n",
    "              ax_sd = sd(ax),\n",
    "              ax_min = min(ax),\n",
    "              ax_max = max(ax),\n",
    "              ax_median = median(ax),\n",
    "              ay_mean = mean(ay),\n",
    "              ay_sd = sd(ay),\n",
    "              ay_min = min(ay),\n",
    "              ay_may = max(ay),\n",
    "              ay_median = median(ay),\n",
    "              az_mean = mean(az),\n",
    "              az_sd = sd(az),\n",
    "              az_min = min(az),\n",
    "              az_maz = max(az),\n",
    "              az_median = median(az),\n",
    "              aT_mean = mean(aT),\n",
    "              aT_sd = sd(aT),\n",
    "              aT_min = min(aT),\n",
    "              aT_maT = max(aT),\n",
    "              aT_median = median(aT),\n",
    "              y = mean(y)\n",
    "             )\n",
    "    features = rbind(features, res)\n",
    "}\n",
    "\n",
    "#view all features\n",
    "features$y = as.factor(features$y)\n",
    "head(features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create sample size for training the model\n",
    "From the information, we know that we have four states in the data. Based on this information, the data is split into train and test samples. The train set is used to build the model and test set is used to validate the model. The ratio between train and test is 80:20. You can adjust this based on type of data. The below table shows the number of observations for each group.   \n",
    "\n",
    "Note: It is adviced to have atleast 30 samples for each group. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T19:23:47.823447Z",
     "start_time": "2019-01-10T19:23:47.767Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\n",
       "  1   2   3   4 \n",
       " 67 109  93  93 "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "table(features$y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the above results, we can observe that there are atleast 30 samples for each group. Now, we can used this data to split into train and test set. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T19:25:44.315656Z",
     "start_time": "2019-01-10T19:25:44.255Z"
    }
   },
   "outputs": [],
   "source": [
    "#create samples of 80:20 ratio\n",
    "sample = sample(nrow(features) , nrow(features)* 0.8)\n",
    "train = features[sample,]\n",
    "test = features[-sample,]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Supervised Fault Classification\n",
    "\n",
    "### Fault Classification using Random Forest\n",
    "\n",
    "Random Forest is a flexible, easy to use machine learning algorithm that produces, even without hyper-parameter tuning, a great result most of the time. It is also one of the most used algorithms, because it’s simplicity and the fact that it can be used for both classification and regression tasks. In this post, you are going to learn, how the random forest algorithm works and several other important things about it [3]. \n",
    "\n",
    "More about Random forest can be learnt here. \n",
    "\n",
    "https://towardsdatascience.com/the-random-forest-algorithm-d457d499ffcd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T18:33:43.977356Z",
     "start_time": "2019-01-10T18:32:30.729Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Parallel Random Forest \n",
       "\n",
       "289 samples\n",
       " 21 predictor\n",
       "  4 classes: '1', '2', '3', '4' \n",
       "\n",
       "No pre-processing\n",
       "Resampling: Cross-Validated (10 fold, repeated 20 times) \n",
       "Summary of sample sizes: 260, 259, 260, 260, 260, 260, ... \n",
       "Resampling results across tuning parameters:\n",
       "\n",
       "  mtry  Accuracy   Kappa    \n",
       "    2   0.9302873  0.9060006\n",
       "   65   0.9795237  0.9725126\n",
       "  128   0.9767511  0.9687818\n",
       "\n",
       "Kappa was used to select the optimal model using the largest value.\n",
       "The final value used for the model was mtry = 65."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#If you don't want parallel computing, comment bellow lines of code. \n",
    "#create parallel clusters for 16 cores\n",
    "cl <- makeCluster(16)\n",
    "registerDoParallel(cl)\n",
    "\n",
    "#set training partameters\n",
    "fitControl = trainControl(method = \"repeatedcv\",\n",
    "                                   number = 10,\n",
    "                                   ## repeated ten times\n",
    "                                   repeats = 20)\n",
    "\n",
    "rf.model = train(y ~ ., train,\n",
    "                         method = \"parRF\", \n",
    "                         allowParallel = TRUE, \n",
    "                         metric =\"Kappa\",\n",
    "                         trControl = fitControl,\n",
    "                         verbose = FALSE\n",
    "                         #rfeControl=control\n",
    "                        )\n",
    "\n",
    "#summary of trained model\n",
    "rf.model\n",
    "\n",
    "#close all clusters\n",
    "stopCluster(cl)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the training results, we can note that 97% accuracy and 96% kappa value has been achieved. This indicates a very good model. Next, we need to validate based on test data set and compute the validation accuracy. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T18:42:16.884635Z",
     "start_time": "2019-01-10T18:42:16.548Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Confusion Matrix and Statistics\n",
       "\n",
       "          Reference\n",
       "Prediction  1  2  3  4\n",
       "         1 12  0  0  0\n",
       "         2  0 20  0  0\n",
       "         3  0  0 21  0\n",
       "         4  0  0  1 19\n",
       "\n",
       "Overall Statistics\n",
       "                                         \n",
       "               Accuracy : 0.9863         \n",
       "                 95% CI : (0.926, 0.9997)\n",
       "    No Information Rate : 0.3014         \n",
       "    P-Value [Acc > NIR] : < 2.2e-16      \n",
       "                                         \n",
       "                  Kappa : 0.9815         \n",
       " Mcnemar's Test P-Value : NA             \n",
       "\n",
       "Statistics by Class:\n",
       "\n",
       "                     Class: 1 Class: 2 Class: 3 Class: 4\n",
       "Sensitivity            1.0000    1.000   0.9545   1.0000\n",
       "Specificity            1.0000    1.000   1.0000   0.9815\n",
       "Pos Pred Value         1.0000    1.000   1.0000   0.9500\n",
       "Neg Pred Value         1.0000    1.000   0.9808   1.0000\n",
       "Prevalence             0.1644    0.274   0.3014   0.2603\n",
       "Detection Rate         0.1644    0.274   0.2877   0.2603\n",
       "Detection Prevalence   0.1644    0.274   0.2877   0.2740\n",
       "Balanced Accuracy      1.0000    1.000   0.9773   0.9907"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Used the model to perform prediction\n",
    "prediction = data.frame(pred=predict.train(rf.model, test))\n",
    "\n",
    "#create a confusion matrix\n",
    "caret::confusionMatrix(test$y, prediction$pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the above results, we can observe that the following model achived highest accuracy. Also, no information rate is less than accuracy indicating that the model is reliable. Also, kappa value is very high indicating similar inference. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fault Classification using Support Vector Machine (SVM)\n",
    "\n",
    "A Support Vector Machine (SVM) is a discriminative classifier formally defined by a separating hyperplane. In other words, given labeled training data (supervised learning), the algorithm outputs an optimal hyperplane which categorizes new examples. In two dimentional space this hyperplane is a line dividing a plane in two parts where in each class lay in either side [4]\n",
    "\n",
    "An indetail tutorial regarding SVM can be found here. \n",
    "https://medium.com/machine-learning-101/chapter-2-svm-support-vector-machine-theory-f0812effc72\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T18:55:16.510445Z",
     "start_time": "2019-01-10T18:54:44.459Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Support Vector Machines with Linear Kernel \n",
       "\n",
       "289 samples\n",
       " 21 predictor\n",
       "  4 classes: '1', '2', '3', '4' \n",
       "\n",
       "No pre-processing\n",
       "Resampling: Cross-Validated (10 fold, repeated 20 times) \n",
       "Summary of sample sizes: 260, 261, 260, 260, 261, 260, ... \n",
       "Resampling results:\n",
       "\n",
       "  Accuracy   Kappa  \n",
       "  0.9580475  0.94358\n",
       "\n",
       "Tuning parameter 'C' was held constant at a value of 1"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#If you don't want parallel computing, comment bellow lines of code. \n",
    "#create parallel clusters for 16 cores\n",
    "cl <- makeCluster(16)\n",
    "registerDoParallel(cl)\n",
    "\n",
    "#set training partameters\n",
    "fitControl = trainControl(method = \"repeatedcv\",\n",
    "                                   number = 10,\n",
    "                                   ## repeated ten times\n",
    "                                   repeats = 20)\n",
    "\n",
    "rf.model = train(y ~ ., train,\n",
    "                         method = \"svmLinear\", \n",
    "                         allowParallel = TRUE, \n",
    "                         metric =\"Kappa\",\n",
    "                         trControl = fitControl,\n",
    "                         verbose = FALSE\n",
    "                        )\n",
    "\n",
    "#summary of trained model\n",
    "rf.model\n",
    "\n",
    "#close all clusters\n",
    "stopCluster(cl)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the above results we can observe that accuracy and kappa values have decreased. But, this decrease is very minor and negligible. Next we can validate with test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T18:56:13.215834Z",
     "start_time": "2019-01-10T18:56:12.949Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Confusion Matrix and Statistics\n",
       "\n",
       "          Reference\n",
       "Prediction  1  2  3  4\n",
       "         1 12  0  0  0\n",
       "         2  0 20  0  0\n",
       "         3  0  0 19  2\n",
       "         4  0  0  0 20\n",
       "\n",
       "Overall Statistics\n",
       "                                          \n",
       "               Accuracy : 0.9726          \n",
       "                 95% CI : (0.9045, 0.9967)\n",
       "    No Information Rate : 0.3014          \n",
       "    P-Value [Acc > NIR] : < 2.2e-16       \n",
       "                                          \n",
       "                  Kappa : 0.963           \n",
       " Mcnemar's Test P-Value : NA              \n",
       "\n",
       "Statistics by Class:\n",
       "\n",
       "                     Class: 1 Class: 2 Class: 3 Class: 4\n",
       "Sensitivity            1.0000    1.000   1.0000   0.9091\n",
       "Specificity            1.0000    1.000   0.9630   1.0000\n",
       "Pos Pred Value         1.0000    1.000   0.9048   1.0000\n",
       "Neg Pred Value         1.0000    1.000   1.0000   0.9623\n",
       "Prevalence             0.1644    0.274   0.2603   0.3014\n",
       "Detection Rate         0.1644    0.274   0.2603   0.2740\n",
       "Detection Prevalence   0.1644    0.274   0.2877   0.2740\n",
       "Balanced Accuracy      1.0000    1.000   0.9815   0.9545"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Used the model to perform prediction\n",
    "prediction = data.frame(pred=predict.train(rf.model, test))\n",
    "\n",
    "#create a confusion matrix\n",
    "caret::confusionMatrix(test$y, prediction$pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the above validation results, we can conclude that the results have marginally decreased compared to random forest model. Accuracy is much higher than NIR which indicates that this is still a reliable model. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fault Classification using K-nearest Neighbour (KNN)\n",
    "KNN is one of the simplest supervised learning method. This model can be used for both classification and regression models. One of the drawbacks of this model is, this is on the fly training model. This is a time consuming model. As the amount of data increases, the training time increases. \n",
    "\n",
    "K nearest neighbors is a simple algorithm that stores all available cases and classifies new cases based on a similarity measure (e.g., distance functions). KNN has been used in statistical estimation and pattern recognition already in the beginning of 1970’s as a non-parametric technique [5]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T19:26:00.556897Z",
     "start_time": "2019-01-10T19:25:49.467Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "k-Nearest Neighbors \n",
       "\n",
       "289 samples\n",
       " 21 predictor\n",
       "  4 classes: '1', '2', '3', '4' \n",
       "\n",
       "Pre-processing: centered (128), scaled (128) \n",
       "Resampling: Cross-Validated (10 fold, repeated 3 times) \n",
       "Summary of sample sizes: 261, 260, 260, 260, 261, 259, ... \n",
       "Resampling results across tuning parameters:\n",
       "\n",
       "  k  Accuracy   Kappa     \n",
       "  5  0.2919359  0.06488064\n",
       "  7  0.7431069  0.65660220\n",
       "  9  0.8702394  0.82535956\n",
       "\n",
       "Accuracy was used to select the optimal model using the largest value.\n",
       "The final value used for the model was k = 9."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#create parallel clusters for 16 cores\n",
    "cl <- makeCluster(16)\n",
    "registerDoParallel(cl)\n",
    "#set training partameters\n",
    "fitControl = trainControl(method = \"repeatedcv\",\n",
    "                                   ## repeated ten times\n",
    "                                   repeats = 3)\n",
    "\n",
    "rf.model = train(y ~ ., train,\n",
    "                         method = \"knn\",\n",
    "                         trControl = fitControl,\n",
    "                         preProcess = c(\"center\",\"scale\")\n",
    "                        )\n",
    "\n",
    "#summary of trained model\n",
    "rf.model\n",
    "\n",
    "\n",
    "#close all clusters\n",
    "stopCluster(cl)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the train results we can note that the accuracy of KNN is much lower than RandomForest and SVM. Next, we need to test the accuracy of validation data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T19:26:42.420509Z",
     "start_time": "2019-01-10T19:26:42.056Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Confusion Matrix and Statistics\n",
       "\n",
       "          Reference\n",
       "Prediction  1  2  3  4\n",
       "         1 12  0  0  0\n",
       "         2  1 19  0  0\n",
       "         3  1  0 15  5\n",
       "         4  0  0  0 20\n",
       "\n",
       "Overall Statistics\n",
       "                                          \n",
       "               Accuracy : 0.9041          \n",
       "                 95% CI : (0.8124, 0.9606)\n",
       "    No Information Rate : 0.3425          \n",
       "    P-Value [Acc > NIR] : < 2.2e-16       \n",
       "                                          \n",
       "                  Kappa : 0.8712          \n",
       " Mcnemar's Test P-Value : NA              \n",
       "\n",
       "Statistics by Class:\n",
       "\n",
       "                     Class: 1 Class: 2 Class: 3 Class: 4\n",
       "Sensitivity            0.8571   1.0000   1.0000   0.8000\n",
       "Specificity            1.0000   0.9815   0.8966   1.0000\n",
       "Pos Pred Value         1.0000   0.9500   0.7143   1.0000\n",
       "Neg Pred Value         0.9672   1.0000   1.0000   0.9057\n",
       "Prevalence             0.1918   0.2603   0.2055   0.3425\n",
       "Detection Rate         0.1644   0.2603   0.2055   0.2740\n",
       "Detection Prevalence   0.1644   0.2740   0.2877   0.2740\n",
       "Balanced Accuracy      0.9286   0.9907   0.9483   0.9000"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Used the model to perform prediction\n",
    "prediction = data.frame(pred=predict.train(rf.model, test))\n",
    "\n",
    "#create a confusion matrix\n",
    "caret::confusionMatrix(test$y, prediction$pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the confusion matrix we can observe that the accuracy and kappa value is significantly lower than the values compared to random forest and SVM model. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fault Classification using AutoML\n",
    "\n",
    "The H2O AutoML interface is designed to have as few parameters as possible so that all the user needs to do is point to their dataset, identify the response column and optionally specify a time constraint or limit on the number of total models trained [6]. \n",
    "\n",
    "The below model will be trained for 30 seconds for this tutorial. This can be scaled up or down based on needs. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T19:56:56.562238Z",
     "start_time": "2019-01-10T19:56:34.824Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Connection successful!\n",
      "\n",
      "R is connected to the H2O cluster: \n",
      "    H2O cluster uptime:         2 days 43 minutes \n",
      "    H2O cluster timezone:       America/New_York \n",
      "    H2O data parsing timezone:  UTC \n",
      "    H2O cluster version:        3.20.0.2 \n",
      "    H2O cluster version age:    8 months and 11 days !!! \n",
      "    H2O cluster name:           H2O_started_from_R_aanamruthn_tdy557 \n",
      "    H2O cluster total nodes:    1 \n",
      "    H2O cluster total memory:   7.48 GB \n",
      "    H2O cluster total cores:    16 \n",
      "    H2O cluster allowed cores:  16 \n",
      "    H2O cluster healthy:        TRUE \n",
      "    H2O Connection ip:          localhost \n",
      "    H2O Connection port:        54321 \n",
      "    H2O Connection proxy:       NA \n",
      "    H2O Internal Security:      FALSE \n",
      "    H2O API Extensions:         Algos, AutoML, Core V3, Core V4 \n",
      "    R Version:                  R version 3.5.1 (2018-07-02) \n",
      "\n",
      "  |======================================================================| 100%\n",
      "  |======================================================================| 100%\n",
      "  |======================================================================| 100%\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "                                               model_id mean_per_class_error\n",
       "1             GBM_grid_0_AutoML_20190227_152539_model_0           0.01250000\n",
       "2                          XRT_0_AutoML_20190227_152539           0.02380952\n",
       "3                          DRF_0_AutoML_20190227_152539           0.03630952\n",
       "4             GLM_grid_0_AutoML_20190227_152539_model_0           0.67857143\n",
       "5 StackedEnsemble_BestOfFamily_0_AutoML_20190227_152539           0.75000000\n",
       "6    StackedEnsemble_AllModels_0_AutoML_20190227_152539           0.75000000\n",
       "     logloss      rmse        mse\n",
       "1 0.06791937 0.1136454 0.01291529\n",
       "2 0.07421694 0.1385458 0.01919493\n",
       "3 0.10305847 0.1574621 0.02479430\n",
       "4 1.30128990 0.7245946 0.52503733\n",
       "5 1.37134105 0.7439822 0.55350945\n",
       "6 1.37134105 0.7439822 0.55350945\n",
       "\n",
       "[6 rows x 5 columns] "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#initialize H2O\n",
    "h2o.init()\n",
    "\n",
    "#change to h2o dataframe.\n",
    "trainAML = as.h2o(train)\n",
    "testAML = as.h2o(test)\n",
    "\n",
    "# Identify predictors and response\n",
    "y = \"y\"\n",
    "x <- setdiff(names(train), y)\n",
    "\n",
    "# For binary classification, response should be a factor\n",
    "trainAML[,y] = as.factor(trainAML[,y])\n",
    "testAML[,y] = as.factor(testAML[,y])\n",
    "\n",
    "#train model using AutoML\n",
    "aml <- h2o.automl(y = y, training_frame = trainAML, leaderboard_frame = testAML, \n",
    "    max_runtime_secs = 10)\n",
    "\n",
    "# View the AutoML Leaderboard\n",
    "lb <- aml@leaderboard\n",
    "lb\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-10T19:57:22.121563Z",
     "start_time": "2019-01-10T19:57:21.868Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  |======================================================================| 100%\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Confusion Matrix and Statistics\n",
       "\n",
       "          Reference\n",
       "Prediction  1  2  3  4\n",
       "         1 12  0  0  0\n",
       "         2  0 20  0  0\n",
       "         3  0  0 21  0\n",
       "         4  0  0  1 19\n",
       "\n",
       "Overall Statistics\n",
       "                                         \n",
       "               Accuracy : 0.9863         \n",
       "                 95% CI : (0.926, 0.9997)\n",
       "    No Information Rate : 0.3014         \n",
       "    P-Value [Acc > NIR] : < 2.2e-16      \n",
       "                                         \n",
       "                  Kappa : 0.9815         \n",
       " Mcnemar's Test P-Value : NA             \n",
       "\n",
       "Statistics by Class:\n",
       "\n",
       "                     Class: 1 Class: 2 Class: 3 Class: 4\n",
       "Sensitivity            1.0000    1.000   0.9545   1.0000\n",
       "Specificity            1.0000    1.000   1.0000   0.9815\n",
       "Pos Pred Value         1.0000    1.000   1.0000   0.9500\n",
       "Neg Pred Value         1.0000    1.000   0.9808   1.0000\n",
       "Prevalence             0.1644    0.274   0.3014   0.2603\n",
       "Detection Rate         0.1644    0.274   0.2877   0.2603\n",
       "Detection Prevalence   0.1644    0.274   0.2877   0.2740\n",
       "Balanced Accuracy      1.0000    1.000   0.9773   0.9907"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pred = h2o.predict(aml@leader, testAML)\n",
    "preds = as.data.frame(pred)\n",
    "#create a confusion matrix\n",
    "caret::confusionMatrix(test$y, preds$predict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model used during AutoML process is GBM. This model provided the highest accuracy during training. From the validation results, we can observe that the accuracy is much higher than NIR and Kappa is also much higher compared to all the other models in this tutorial. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### References\n",
    "\n",
    "[1] Amruthnath, Nagdev, and Tarun Gupta. \"A research study on unsupervised machine learning algorithms for early fault detection in predictive maintenance.\" In 2018 5th International Conference on Industrial Engineering and Applications (ICIEA), pp. 355-361. IEEE, 2018.\n",
    "\n",
    "[2] Amruthnath, Nagdev, and Tarun Gupta. \"Fault class prediction in unsupervised learning using model-based clustering approach.\" In Information and Computer Technologies (ICICT), 2018 International Conference on, pp. 5-12. IEEE, 2018.\n",
    "\n",
    "[3] Niklas Donges, \"The Random Forest Algorithm\", https://towardsdatascience.com/the-random-forest-algorithm-d457d499ffcd. \n",
    "\n",
    "[4] Savan Patel, \"Chapter 2 : SVM (Support Vector Machine) — Theory\", https://medium.com/machine-learning-101/chapter-2-svm-support-vector-machine-theory-f0812effc72\n",
    "\n",
    "[5] \"K Nearest Neighbors - Classification\", https://www.saedsayad.com/k_nearest_neighbors.htm\n",
    "\n",
    "[6] \"AutoML: Automatic Machine Learning\", http://docs.h2o.ai/h2o/latest-stable/h2o-docs/automl.html"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
